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Abstract 

In  the  highly  dynamic  and  information  rich  environment  of  military  submarine  operations, 
the  Commanding  Officer  must  make  split  second  decisions  that  could  ultimately  result  in 
the  destruction  or  survival  of  ownship  or  the  accomplishment  of  its  designated  mission. 
Despite  extensive  training  and  expertise,  the  fog  of  war  often  shadows  real  world 
encounters.  Pieces  of  the  tactical  puzzle  may  be  scattered,  broken,  and/or  even  missing. 

It  is  the  Commanding  Officer’s  job  to  meld  the  fog  into  a  clear  tactical  picture  and 
intuitively  decide  upon  the  optimal  course  of  action  to  obtain  his  mission  goals  and 
objectives.  A  mathematically  derived  optimal  solution  could  be  used  as  a  decision  making 
aid  for  the  Commanding  Officer  to  enhance  ownship’ s  performance.  A  highly  innovative 
hybrid  Bayesian/differential  game  modeling  approach  is  being  developed  utilizing  21CSI’s 
Agent  Enhanced  Decision  Guide  Environment  (AEDGE™),  a  COTS  based  agent 
architecture,  to  tackle  this  problem.  The  resulting  decision  aid  will  facilitate  submarine 
operations  by  addressing  such  practical  problems  as  tradeoff  evaluation  for  course  of 
action  and  maintaining  tactical  advantage  while  avoiding  counter  detection. 

1  Introduction 

The  task  of  providing  accurate,  timely,  and  focused  decisions  in  submarine  tactical  command  and 
control  environment  is  a  complicated  problem.  Difficulties  arise  from  the  inherent  uncertainties 
embedded  in  the  situations,  such  as  the  imprecision  of  the  sensory  data,  the  unpredictability  of  the 
Contact's  tactical  picture,  and  the  inadequacy  of  the  decision  model  and  reasoning  methodology. 
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Action  determination  is  performed  in  a  complex  decision  space  consisting  of  a  large  parameter  set 
that  includes  the  evaluation  criteria,  constraints,  and  action  alternatives.  In  most  situations, 
neither  the  sets  of  available  information  are  complete,  precise,  and  consistent,  nor  would  the 
decision  space  and  its  parameter  set  be  defined  unambiguously. 

In  the  highly  dynamic  and  information  rich  environment  of  military  submarine  operations,  the 
Commanding  Officer  (CO)  must  make  split  second  decisions  that  could  ultimately  result  in  the 
destruction  or  survival  of  Ownship  or  the  accomplishment  of  its  designated  mission.  Despite 
extensive  training  and  expertise,  the  fog  of  war  often  shadows  real  world  encounters.  Pieces  of 
the  tactical  puzzle  may  be  scattered,  broken,  and/or  even  missing.  It  is  the  CO’s  job  to  meld  the 
fog  into  a  clear  tactical  picture  and  intuitively  decide  upon  the  optimal  course  of  action  to  obtain 
his  mission  goals  and  objectives.  The  problem  always  comes  under  the  assumption  that  the  CO 
and  his  staff  have  certain  amount  of  information  about  the  contact  and  limited  knowledge  for 
modeling  the  action/reaction  of  the  contact.  But  the  information  never  comes  to  be  complete 
and/or  hundred  percent  accurate.  When  under  uncertainty,  the  system  needs  to  take  account  of 
how  to  switch  between  actions  and  response  to  new  information,  to  retract  actions,  and  to  regain 
control  points.  How  do  they  properly  make  maximum  use  of  the  information  and  how  do  they 
make  decisions  based  on  the  most  effective  use  of  the  information?  It  is  desirable  to  have  a 
computer  system  automatically  process  the  collected  information,  evaluate  the  alternatives  and 
provide  on-time  assistance  to  the  CO  for  making  the  decisions. 

A  great  deal  of  work  has  been  devoted  to  the  study  of  the  nature  and  formalism  of  various 
decision  models  and  quantitative  reasoning  techniques  such  as  the  differential  game  theory, 
Bayesian  statistics,  and  utility  economics,  to  taper  off  the  uncertainty  problems  in  decision 
making.  However,  most  studies  have  concentrated  on  individual  models  and  techniques.  Little 
has  been  done  to  unify  them  coherently  to  provide  a  collective  solution.  This  paper  explores  the 
problem  from  the  perspective  of  two  fundamental  theories  and  their  complementary  roles  in 
decision  science:  differential  game  theory  and  classic  Bayesian  probability  theory. 

A  mathematically  derived  optimal  solution  could  be  used  as  a  decision  making  aid  for  the  CO  to 
enhance  ownship’s  performance.  Specifically,  it  is  known  that 

•  The  use  of  computer  technology  affords  a  rapid  and  efficient  manipulation  and  exploration  of 
enormous  amounts  of  available  information  to  deliver  real  time  optimal  strategies  in  a  dynamic 
environment. 

•  The  enormous  storage  capabilities  available  in  today’s  computer  systems  allow  processing  of 
historical  data  and  prior  experience  as  no  human  can. 

•  The  advances  in  game  theory,  artificial  intelligence  and  software  agent  technology  allows  for 
drawing  optimal  solutions  available  to  the  CO  in  real  time. 

•  Using  sufficient  knowledge  and  feasibility  estimations  in  heuristic  reasoning,  the  decision  aid 
can  determine  up-front  whether  certain  objectives  can  be  fulfilled. 

A  COTS  based  agent  architecture,  21CSI’s  Agent  Enhanced  Decision  Guide  Environment 
(AEDGE™),  will  be  utilized  to  implement  the  decision  aid  methodology.  The  computer  decision 
support  system  will  determine  the  optimal  strategy  and  a  set  of  actions  for  unit  commander  to 
choose  after  receiving  and  collecting  necessary  information  from  various  resources  (from  both 
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own  side  and  the  adversaries).  The  knowledge  acquisition  and  decision  support  tool  will  provide 
advice  and  a  coherent  plan  of  actions  to  the  Commanding  Officer  (CO)  under  the  concerns  of  the 
following  uncertainties:  (1)  measurement  uncertainty  which  comes  from  sensor  noises, 
environment  dynamics  and  complexity  in  terms  of  the  presence  of  different  parties  in  the  area,  the 
limitations  of  sensor  detecting  ranges,  etc.;  and  (2)  unpredictability  of  the  contact’s  actions  -  the 
contact  may  unaware  of  the  presence  of  own  unit.  In  case  it  knows  us,  it  may  be  able  to  estimate 
and  predict  our  moves,  and  take  the  best  counter  action  toward  us,  while  the  own  unit  makes 
estimation  and  predictions  of  the  contact’s  move  and  actions. 

In  the  following,  we  will  first  present  an  overview  of  game  theory  based  decision  aid  systems  and 
techniques  that  address  the  problems  of  tactical  decision  aid  under  overwhelming  information 
inflow  and  uncertainty  in  section  II.  The  principles  and  computational  details  of  the  differential 
games  and  Bayesian  statistics  for  tactic  command  and  control  environment  are  then  described  in 
section  III.  The  theoretical  discussions  are  assisted  with  example  illustrations.  The  AEDGE™ 
architecture  and  its  application  in  implementing  the  decision  aid  model  are  described  in  section  IV. 
We  use  diagrams  and  examples  to  illustrate  the  coherent  relations  among  the  software  modules 
and  the  functionalities  each  of  the  modules  will  play  in  terms  of  their  implementation  of  the  mixed 
strategy.  The  section  V  briefly  summarizes  the  major  technique  points  of  the  paper. 

2  Game  theoretic  decision  aid 

A  well-crafted  computer  software  system  can  assist  military  planners  in  their  tactical  decision¬ 
making,  particularly  with  respect  to  quickly  identifying  responses  and  counter-responses  to  enemy 
action  or  inaction.  Unit  commanders  would  apply  such  a  tool  in  order  to  determine  the  best 
allocation  of  tactical  resources  to  accomplish  the  unit  mission  and  satisfy  the  commander’s  intent. 
When  the  staff  uses  a  suitably  automated  “war  gaming”  tool  to  support  Course  of  Action  (COA) 
analysis,  the  unit  commander  can  quickly  gain  a  comprehensive  understanding  of  the  action- 
counteraction  dynamic  between  the  opposing  units. 

The  term  "Differential  Games"  is  applied  to  a  group  of  problems  in  applied  mathematics  that  share 
certain  characteristics  related  to  the  modeling  of  conflict  [Isaacs  1965].  In  a  basic  differential 
game’s  setting  there  are  two  actors  -  a  pursuer  and  an  evader  -  with  conflicting  goals.  The  pursuer 
wishes,  in  some  sense,  to  catch  the  evader,  while  the  evader's  mission  is  to  prevent  this  capture. 
These  "games"  are  modeled  mathematically  by  first  defining  state  variables  that  represent  the 
position  (and  perhaps  velocity)  of  the  participants,  determining  (differential)  equations  of  motion 
for  the  rivals,  and  then  describing  sets  in  the  state  space  called  target  sets.  (For  example,  a  target 
set  for  a  pursuer  may  include  points  in  the  state  space  where  the  distance  between  the  pursuer  and 
the  evader  is  small.)  Each  participant  in  the  game  tries  to  drive  the  state  variables  of  the  game  into 
a  particular  target  set  by  controlling  key  variables  called,  naturally,  controls.  The  study  of  these 
games  has  implications  for  real-life  air  combat. 

Pursuit-evasion  games  have  been  applied  in  modeling  predator  chasing  prey,  a  missile  chasing  an 
aircraft,  or  the  like  [Hajek  1975,  Petrosjan  1993].  Unlike  most  other  games,  the  players  may  have 
to  make  continuous  decisions,  for  example  deciding  on  a  real-valued  turn  angle  at  every  moment 
in  time.  Thus,  it  has  a  feature  of  dynamical  games.  An  archetypal  example  of  a  differential  game  is 
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known  as  the  Homicidal  Chauffeur  [Isaacs  1965].  In  this  game,  the  driver  of  a  circular  car  acts  to 
knock  down  a  pedestrian,  who,  of  course,  does  not  wish  to  be  flattened.  The  car  can  move  faster 
than  the  pedestrian,  but  the  pedestrian  can  maneuver  better.  What  is  the  best  strategy  for  the 
pursuer  (the  car)  and  the  evader  (the  pedestrian)  to  follow  in  order  for  each  to  achieve  their 
conflicting  goals?  The  situation  of  submarine  in  a  battle  space  is  the  same  as  this  game  model. 
The  solution  to  this  problem  can  be  applied  to  air  combat  where  a  slow,  but  more  maneuverable 
airplane  is  pursued  by  a  faster  but  less  maneuverable  craft. 

We  will  limit  our  discussion  to  a  two-player  game  problem  in  this  presentation.  It  is 
straightforward  to  extend  it  to  multi-player  problems.  In  fact,  in  most  research  and  practical  study, 
the  multi-player  game  problem  is  often  treated  simply  as  multiple  two-player  game  problems.  To 
better  understand  the  principles  and  methods  behind  the  system  structures  of  the  military  tactic 
decision  making  system,  we’d  like  to  first  recognize  some  specific  features  described  in  terms  of 
differential  games  paradigm.  The  features  include  the  following: 

(1)  Dynamic  game  -  The  process  involves  actions  and  counter- actions  from  adversary  players. 
That  is,  it  is  generally  a  multi-player,  multi-objectives,  multi-variables,  and  adversary  game 
problem. 

(2)  Continuous  action  game  -  That  is,  the  players  involved  in  the  game  are  committed  to  adopting 
and  adjusting  actions  all  the  time.  Also,  the  game  actions  are  nomonotonic  -  players  are 
allowed  to  alter  actions  any  time  upon  receiving  new  information. 

(3)  Evolutionary  game  -  There  will  be  game  state  changes  continuously  and  be  infinite  number  of 
possible  states  in  long  run. 

(4)  Simultaneous  game  -  the  players  may  have  or  have  no  knowledge  about  the  identity  and 
actions  of  each  other  while  making  their  own  decisions.  That  is,  one  player  may  make  a 
decision  based  on  the  information  and  the  (estimated)  action  made  by  the  other  player  at  the 
same  time  or  after  the  other  player  takes  an  action. 

(5)  Non-deterministic  game  -  Generally,  information  available  to  the  players  are  uncertain 
(especially  the  opponent’s  plan,  action,  reaction,  etc.),  incomplete,  and  dynamic.  However, 
the  certainty  of  information  improves  when  stayed  longer  and  have  done  more  measurements. 
For  example,  the  velocity  of  contact  and  the  prediction  of  the  trajectory  are  uncertain  on  the 
first  few  attempts  of  measurement  and  prediction;  but  the  certainty  increases  (uncertainty 
decreases)  when  more  and  more  measurements  and  predictions  are  made. 

(6)  Informed  game  -  There  exists  information  exchange,  detection,  and  prediction  between 
players.  In  general  a  player  makes  decision  based  on  the  combined  information  of  (a) 
environment  and  sensor  data,  (b)  acquired  facts  of  the  other  player’s  previous  action,  and  (c) 
estimated  facts  and  prediction  of  other  player’s  current  action. 

(7)  Game  with  incomplete  information:  The  players  are  uncertain  about  some  important 
parameters  of  the  game  situation,  such  as  the  payoff  functions  of  the  opponent  or  even  the 
player  themself,  the  physical  facilities  and  strategies  available  to  other  players  or  even  to 
themselves,  the  information  other  players  have  about  the  various  aspect  of  the  game  situation, 
etc. 

(8)  Bayesian  Games:  each  play  has  a  subjective  probability  distribution  over  the  alternative 
possibilities.  The  idea  of  Bayesian  game  is  to  construct,  for  any  information-incomplete  game 
G,  some  information-complete  game  G*  that  are  game-theoretically  equivalent  to  G  [Harsanyi 
1967-1968], 
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The  analysis  of  these  features  helped  us  to  figure  out  a  decision-aid  model  and  design  strategy  that 
address  the  tactic  decision  aid  problems  possessing  these  features.  Such  decision  aid  system  in 
tactic  decision  aid  is  based  on  the  theory  and  models  of  dynamic  continuous-time  infinite  dynamic 
games.  The  multi-variable,  multi-objective  optimization  model  is  rooted  (started)  from  the 
development  of  optimization  theory  during  the  World  War  II.  As  indicated  by  Basar  [Basar 
1982],  the  term  “differential  game”  became  a  general  name  for  games  where  differential  equations 
play  an  important  role.  The  term  is  also  being  used  for  other  classes  of  games  constituting  a  class 
of  decision  problems  where  the  evolution  of  the  states  is  described  by  a  differential  equation  or 
some  other  forms  of  dynamic  system  representation  and  the  players  act  through  a  time  interval. 

The  differential  game  theory  models  the  submarine  tactic  decision  making  situations  by  a  set  of 
differential  equations  that  describe  the  objectives  and  constraints  that  are  solvable  by  the 
traditional  optimization  methods  [Ghosh  1998].  The  elegancy  of  the  differential  game  theory  lies 
on  the  Saddle-point  equilibrium  principle  (the  principle  of  minimax  game  theory)  [Gintis  2000]. 
The  principle  states  that  the  fundamental  of  game  theory  is  (for  a  player  A)  to  choose  the  move 
(action)  such  that  its  maximum  loss  (loss  ceiling)  is  no  bigger  than  that  (the  maximum  loss)  of  the 
other  alternatives  under  the  assumption  (condition)  that  the  other  player  is  to  adopt  the  best 
counter-action  against  player  A’s  move.  Or,  in  other  way  of  saying  this,  choose  the  move  that  the 
minimum  gain  (gain-floor)  is  no  smaller  than  that  (the  minimum  gain)  of  any  other  alternatives  (no 
matter  how  large  the  maximum  gain  would  be  with  the  other  alternatives)  under  the  condition  that 
the  other  player  is  to  adopt  the  best  counteraction  against  player  A.  Specifically,  the  differential 
game  model  for  the  submarine  tactic  decision  aid  consists  of  the  following  components. 

(1)  two  players  (ownship  and  the  contact)  that  form  the  two  sides  of  the  game  in  the  game, 
denoted  as  Pi  and  P2. 

(2)  a  time  interval  [0,  T]  within  which  continuous  decisions  (game  playing)  are  to  be  made. 

(3)  an  infinite  set  of  states  s  =  (x(t)},  where  t  e  [0,  T],  denotes  the  environment.  A  sequence  of 
states  s°,  s1,  ...,  sq  is  a  state  trajectory  that  depicts  possible  consequences  when  an  action 
(decision)  is  made. 

(4)  a  state  value  (evaluation)  function  for  each  player,  denoted  as  T|i(s'),  and  r)2(s1). 

(5)  a  definite  set  of  action  (control)  alternatives  for  each  player,  denoted  as  Ui‘  =  {ui‘(t)}  and  u2‘  = 
{ u2‘(t) } ;  i  =  1,  2,  ...,  c. 

(6)  a  cost  function  for  each  player:  L1;  L2  =  g(t,  x(t),  u2'(t),  u22(t),  ...,  u2c(t)). 

(7)  a  differential  equation  — =/( t,  x(t),  m'/t),  u->2(t),  ...,  u7c(t))  describes  the  state  changes 

dt 

(evolution)  of  the  game. 

(8)  a  mapping  u‘(t)  =  r‘(t,  x(t))  is  a  selection  of  a  permissible  strategy  (action  alternatives). 

(9)  information  structure  -  a  set  of  information  fields  that  are  accessible  by  pi  and/or  p2. 

The  principle  in  tactic  decision  aid  is  to  maximize  the  ownship’ s  prioritized  objective  function 
with  minimum  cost  by  taking  considerations  of  any  counter-actions  the  other  party  may  take.  The 
decision  making  model  prefers  the  setting  of  the  cost  function  properly  so  that  it  leads  to  a  stable 
Nash  solutions.  The  principle  can  be  illustrated  by  an  example  below. 
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Let  us  use  a  chase  and  detection  example  to  illustrate  the  processes  involved  in  the  differential 
game.  Consider  a  situation  in  the  Initial  Detection/Collision  Avoidance  State,  where  the  ownship 
is  to  chase  a  hostile  while  avoiding  being  detected  by  the  contact.  For  the  two  players  (ownship 
and  the  contact)  involved,  we  use  Pi  to  denote  the  ownship  and  P2  for  the  contact.  The  time 
interval  parameter  T  is  defined  as  to  when  the  mission  of  P!  is  called  off.  The  state  set  consists  of 
vectors  x(t)  =  [d(t),  COi,  v1;  0)2,  v2,  bj,  b2],  that  describe  the  distance  d(t)  between  Pi  and  P2,  the 
maximum  angular  velocity  co,  and  the  constant  forward  speed  v,  (i  =  1 ,  2)  of  P,  and  P2,  and  the 
status  (bj,  b?)  of  P,  and  P2  being  detected  by  each  other,  respectively.  The  state  evaluation 
function  for  player  Pi,  i.e.,  T|i(s‘),  is  defined  as  a  procedure  responsible  for  calculating  the  values 
of  d(t),  Ob  and  v2  from  sensory  data  and  deriving  the  bi  and  b?  values.  The  set  of  action  (control) 
alternatives  for  each  player  is  the  rudder  angle  Ui‘(t)  and  u2'(t);  i  =  1,  2,  ...,  c.  The  cost  function 
can  be  described  in  terms  of  the  distance  between  Pi  and  P2,  as  a  function  of  collision-free  and 
detectibility.  In  a  simplified  case  considering  only  a  two  dimensional  space,  it  is  Li  =  r  =  V(  x  1 2  + 
x22)  and  r  <  =  rm,  where  rm  is  the  minimum  distance  between  Pi  and  P2  that  will  keep  the  Pi  in 
tackle  with  P2  while  not  being  detected  or  colliding  with  P2. 

The  differential  equations  describe  the  kinematics  of  the  ownship  are  given  as  the  following: 

.&i  =  -  cOiu'xt  +  v2sin0, 

■&2  =  -  Vi  +  tOiu'xo  +  V2COS0, 

^  =  Obu2  -  COiU1. 

The  differential  equations  describing  the  kinematics  of  the  contact  are  identical  to  that  of  ownship, 
that  is,  each  player  is  characterized  by  two  parameters:  the  maximum  angular  velocity  cq  and  the 
constant  forward  speed  Vj.  The  control  of  P,  is  u1,  which  is  the  rudder  angle,  and  is  bounded  by 
lu'(t)l  <1.  A  number  of  state  trajectory  diagrams  can  be  constructed  for  different  sets  of  parameter 
values  (different  speed  ratios  and  different  maneuvering  capabilities)  to  gain  certain  insights  into 
chasing  and  detection  sensitivity  with  respect  to  these  parameters.  The  target  set  of  the  solution 
can  then  by  described  by  a  cylinder  space  x,2  +  x22  =  rm2,  (recall  rm  is  the  minimum  distance 
between  Pi  and  P2  that  will  keep  the  Pi  in  tackle  with  P2  while  not  being  detected  or  colliding  with 
P2).  A  mapping  u'(t)  =  r'(t,  x(t))  takes  place  where  a  specific  rudder  angle  is  selected  as  a 
permissible  strategy  of  the  equations  under  the  given  state  x(t)  with  respect  to  the  available 
information  structure  of  pi.  This  simple  model  may  not  be  (too)  unreasonable  if  only  short 
duration  maneuvers  are  considered,  during  which  the  players  cannot  change  their  speed  markedly. 

Though  being  attractive  by  its  traditional  and  mathematical  elegancy,  the  differential  game  theory 
has  certain  limitations  in  real  world  applications.  Basically,  the  complexity  of  real  world  situations 
(which  include  both  the  nature  and  human  factors)  precludes  the  establishment  of  a  sufficiently 
accurate,  reasonably  sized,  and  moderate  level  of  complex  equations  in  terms  of  computation  time 
and  space  requirements.  Many  times,  an  analytical  form  of  the  equation  used  in  a  practical 
application  is  over  simplified.  To  be  realistic,  it  will  be  getting  very  complex,  and  very  hard  or 
almost  impractical  to  obtain.  That  is,  the  equations  may  not  be  adequately  manageable.  The 
unrealistic  over-simplification  of  the  differential  equation  makes  the  decision  based  on  this  model 
come  with  large  deviation  off  the  target.  Moreover,  the  implementation  of  the  differential  games 
theory  relies  on  the  satisfaction  of  the  conditions  that  there  exist  unique  solutions  to  the 
differential  equations  that  formulate  the  state  transitions  of  the  game  process  accurately.  These 
conditions  may  not  be  met  in  practical  situations. 


6 


Many  techniques  have  been  suggested  and  studied  to  address  and/or  compensate  the  limitations  of 
the  basic  differential  game  model.  In  this  paper,  we  describe  an  approach  that  is  to 

(1)  use  a  computational  data  structure,  such  as  Bayesian  probability  representation  for 
description  of  probabilistic  relations  of  the  decision  states,  rather  than  an  explicit  and  analytic 
form  of  the  differential  equations. 

(2)  focus  the  state  and  cost  evaluations  on  use  of  heuristic  methods.  It  is  to  pick  the  action 
alternative  that  is  most  likely  (reliably)  to  achieve  an  objective  with  high  priority. 

(3)  to  have  multiple  decision  models  run  simultaneously  and  compete  cooperatively  to  selecting  a 
solution. 

(4)  to  apply  idea  of  dynamical  switching  among  different  decision  models  according  to 
information  gathered,  as  well  as  estimations  and  predictions  made  in  terms  of  the  stochastic 
models  of  the  game. 

3  Hybrid  model  of  decision  aid 

It  is  known  that  the  traditional  deterministic  model  of  reasoning  for  decision  making  bases  on 
rigorous  mathematical  logic  and  indubitable  data  analysis  has  very  limited  ability  to  operate 
effectively  in  those  circumstances  such  as  the  military  tactic  command  and  control  operations  that 
feature  extensive  underlying  uncertainties.  To  have  a  reliable  and  clear  mind  setting  in  the  absence 
of  clairvoyance,  a  computerized  decision  aids  system  needs  to  acknowledge,  represent  and 
incorporate  measures  of  uncertainty,  quantitatively  and  qualitatively,  and  to  employ  human-like, 
stochastic  and  non- monotonic  inferences  [Doyle  1999]. 

Uncertainties  in  decision  aids  come  from  several  aspects.  Specifically,  they  include:  (a)  Action 
alternatives  are  not  crisply  distinguishable  (often  a  broad  outline),  (b)  Evaluation  criteria  are 
vaguely  defined,  (c)  Constraints  are  ambiguous  or  contradictory,  (d)  Evidence  is  incomplete, 
imprecise  and  inconsistent,  (e)  Fitness  of  the  reasoning  mechanism  to  the  underlying  nature  of  the 
problem  is  unknown,  and  (f)  Variation  of  the  system  identifications  is  constant  due  to  time  and 
situation  changes.  A  difficult  challenge  for  the  decision  support  system  development  is  that  the 
system  designer  must  strike  an  appropriate  balance  between  representing  pertinent  game  aspects 
while  abstracting  away  irrelevant  detail  in  order  to  achieve  the  efficiencies  required  to 
appropriately  sample  the  action-reaction  game  space.  In  other  words,  we  can’t  have  a  computer 
model  that  is  so  detailed  that  it  can  only  game  a  few  scenarios  when  thousands  of  scenarios  may 
need  to  be  sampled.  Likewise,  the  game  must  be  designed  in  sufficient  detail  to  provide  useful 
insight  to  the  allocation  of  resources.  In  order  to  strike  this  balance  between  detail  and  abstraction 
the  game  designer  must  have  an  extraordinary  understanding  of  the  mission  related  information 
(Mission,  Enemy,  Troops,  Terrain,  Time,  and  Politics)  under  varying  circumstances. 
Unfortunately,  this  dynamic  is  so  context-dependent  that  units  typically  do  not  learn  the  rules  until 
they  are  already  deployed  to  the  mission.  Current  research  in  this  field  indicates  that  the  best 
solution  to  this  dilemma  is  to  develop  a  simple  knowledge  acquisition  system  that  enables  the 
military  analysts  to  quickly  encode  knowledge -based  rules  into  a  decision  support  war  gaming 
system  after  already  having  deployed  to  the  mission  site.  The  unit  personnel  must  be  able  to  do 
this  without  assistance  of  expensive  contracting  to  civilian  knowledge  engineers  and  scientists. 
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The  benefits  of  using  Bayesian  probabilities  to  model  uncertainty  in  decision  support  are  well 
known  [Pearl  1988,  desJarins  1993].  The  Bayesian  probabilities  capture  many  stochastic-process 
factors.  It  can  be  used  to  predict  the  effects  that  changes  of  certain  attributes  have  on  decision 
processes.  For  example,  consider  a  Commanding  Officer  (CO)  deciding  how  to  act  on  a  reported 
suspicious  contact.  Suppose  that  a  particular  maneuver,  if  taken,  will  provide  the  sonar  systems 
with  an  opportunity  to  classify  the  contact  decisively.  Further  suppose  that  if  the  maneuver  is 
indeed  taken,  the  contact  will  likely  prove  to  be  harmless  with  a  large  probability  value  and  hostile 
with  a  small  probability  value.  In  the  latter  case,  either  the  action  will  force  the  enemy  to  turn 
away  (with  a  very  large  probability  value)  or  the  enemy  will  force  the  ownship  to  run  away  (with  a 
very  small  probability  value).  It  is  likely  that  the  CO  will  conclude  that  the  only  negative  of  the 
possible  outcomes  is  not  very  likely  (very  small  probability  value)  and  indeed  chose  to  undertake 
the  maneuver.  Now,  consider  the  exact  same  probabilities,  with  raised  stakes.  Specifically,  the 
maneuver  will  indeed  provide  a  definitive  classification,  with  the  probability  values  for  the  contact 
being  harmless  and  hostile  being  specifically  defined.  Clearly  the  cases  are  not  quite  tenable  within 
the  framework  of  differential  equations  here.  The  situations,  however,  can  be  effectively  recorded 
in  a  Bayesian  probability  representation  and  efficiently  evaluated. 

The  Bayesian  approach  has  three  distinct  advantages  over  its  constraint-based  counterpart:  (1) 
conclusions  derived  from  the  Bayesian  approach  are  not  susceptible  to  incorrect  categorical 
decisions  about  independent  facts  that  can  occur  with  data  sets  of  finite  size,  (2)  using  the 
Bayesian  approach,  finer  distinctions  among  model  structures  -both  quantitative  and  qualitative- 
can  be  made,  and  (3)  information  from  several  models  can  be  combined  to  make  better  inferences 
and  to  better  account  for  modeling  uncertainty.  Bayesian  probability  enables  reasoning  under 
uncertainty  and  combines  the  advantages  of  an  intuitive  representation  with  a  sound  mathematical 
basis.  There  are  several  ways  of  determining  the  probabilities  for  the  entities  in  the  Bayesian 
probabilities  [Hanks  1994].  One  common  approach  is  to  use  the  joint  probability  distributions  of 
the  constituent  components  such  that 

i 

pr(yk)  =  n  pr(y,k  I  Jtik); 

i=l 

where  yk  is  a  random  variable  representing  the  truth  value  of  a  proposition  (a  belief  or  an  action 
alternative).  The  yiks  are  the  possible  states  of  yk,  and  7liks  are  possible  states  of  a  set  of 
independent  random  variables  yk.  One  of  the  benefits  of  Bayesian  probability  stems  from  the  fact 
that  it  is  able  to  accommodate  both  subjective  probabilities  (elicited  from  domain  experts)  and 
probabilities  based  on  objective  data. 

A  Bayesian  reasoning  process  can  be  initialized  with  certain  background  knowledge  either 
manually  or  automatically  extracted  from  certain  information  sources.  The  attributes  of  relevant 
data  objects  and  the  relations  are  explored  in  a  decision  support  process  using  Bayes’  rule: 

pr(ykmk,4)=MntizAMzA); 

pr(Il  k  1^) 

where  nk  =  { JCik,  7t2k,  ...,  7t|k]  and  4  denotes  the  background  knowledge.  The  Bayesian 

probability  can  readily  handle  missing  data  and  avoid  over-fitting  of  data  in  a  decision-making 
process.  The  processing  of  information  with  multiple  uncertain  resources  can  be  effectively 
handled  by  applying  the  Dempster-Shafer’s  rule  of  evidence  combination  [Bogler  1987,  Baenett 
1991].  It  gives  us  a  basic  computational  scheme  for  evaluating  the  certainties  of  data  sets  from  multiple 


information  sources  and  feeding  them  into  the  Bayesian  probability  computations  for  evaluating  and 
selection  of  action  alternatives  in  a  submarine  tactic  decision  process. 

We  illustrate  the  Bayesian  probabilities  in  the  framework  of  state  evaluations  in  differential  games 
by  continuing  the  use  of  the  example  situations  described  above.  That  is,  consider  the  problem  of 
ownship  chasing  a  hostile  submarine  while  avoiding  being  detected  in  an  open  sea  area.  Let’s  set 
the  situation  to  the  track  submarine’s  navigation  speed  in  the  course  of  tracking.  The  situation  can 
be  described  by  a  decision  tree  as  the  one  depicted  in  figure  1. 


Pr(I  I  ah  a2)  0.9  0.8  0.6  0.6  0.5  0.3  0.6  0.4  0.1 

Figure  1.  A  decision  tree  model  for  probabilistic  evaluation  of  game  states 

In  the  figure,  symbols  “u”,  “s”  and  “d”  stand  for  speeding  “up”,  “stay  the  same”,  and  “down” 
respectively.  The  Pr(C  I  ai,  a2)  gives  an  estimation  of  the  probability  value  such  that  the  two  units 
may  collide  (or  been  detected  by  the  contact)  based  on  their  combined  actions  ai  and  a2  in  current 
situation,  where  ai,  a2  <=  {“u”,  “s”,  “d”}.  The  probability  value  is  calculated  by  a  pre-defined 
evaluation  function  T|i(t,  x(t))  in  the  differential  game  model.  The  Pr(I  I  ai,  a2)  gives  an  estimation 
of  the  probability  value  that  the  ownship  will  still  keep  in  tackle  with  the  contact  based  on  the 
given  actions  ai,  a2  and  a  pre-defined  evaluation  function  T|j(t,  x(t)).  One  possible  solution  for  the 
ownship  to  decide  in  above  situation  is  to  chose  an  action  among  the  three  alternative  such  that  it 
will  maximize  the  probability  of  keep  in  tackle  with  the  contact  and  minimize  the  probability  of 
leading  to  a  collision  (or  been  detected). 

To  make  a  probabilistic  evaluation  of  action  alternatives,  a  number  of  probability  values  need  to 
be  known  or  obtained  before  hand.  A  learning  and  adaptive  module  will  fulfill  this  task.  The 
probabilities  could  be  presented  in  the  form  such  as  “the  probability  of  recommending  action  A 
under  the  conditions  that  ....  “  and  “  the  probability  of  achieving  objective  X  under  the  conditions 
of  specific  objective  priority.”  The  Bayesian  probability  operators  of  the  system  tries  to  answer 
these  questions  by  an  outcome -based  evaluation  -  probability  estimation  based  on  multiple 
possible  outcomes.  The  computation  is  described  as  the  follows. 

Let’s  assuming  that  there  are  equal  chances  for  both  ownship  and  the  contact  to  choose  an  action 
among  the  three  alternatives.  That  is  Pr(ui)  =  Pr(si)  =  Pr(di)  =  1/3  and  Pr(u2)  =  Pr(s2)  =  Pr(d2)  = 
1/3  ;  where  Pr(ui),  Pr(si),  and  Pr(di)  stand  for  the  probability  that  the  wonunit  will  choose  the 
actions  of  “u”,  “s”  and  “d”  respectively.  The  Pr(u2),  Pr(s2)  and  Pr(d2)  are  the  corresponding 
probabilities  for  the  contact.  One  critical  problem  in  the  ownship’ s  decision  making  is  to  predict 
the  counter  action  of  the  contact.  This  uncertainty  factor  can  be  well  represented  by  probability 
estimates  Pr(a2 1  ai),  where  ai  stands  for  an  action  chosen  by  ownship  and  a2  the  counter  action  by 
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the  contact.  For  example,  we  may  have  the  probabilities  Pr(a2  I  ai)  from  an  estimation  or  previous 
experience,  as  listed  in  the  table  below. 


Pr(a2 1  ax) 

ai  =  ui 

ai  =  si 

ai  =  di 

a2  =  u2 

0.9 

0.5 

0.1 

a2  =  s2 

0.5 

0.9 

0.5 

a2  =  d2 

0.5 

0.2 

0.75 

The  Bayes’  probability  rule  stipulates  the  following  computations. 

Pr(C,  Ui,  u2)  =  Pr(C  I  Ui,  u2)  Pr(ui,  u2)  =  Pr(C  I  Ui,  u2)  Pr(u2 1  Ui)  Pr(ui) 
=  0.9  x  0.9  x0.33  =  0.2673 

Pr(C,  dh  s2)  =  Pr(C  I  di,  s2)  Pr(di,  s2)  =  Pr(C  I  du  s2)  Pr(s2 I  dj)  Pr(di) 

=  0.3  x  0.5  x0.33  =  0.0495 

Pr(I,  Ui,  u2)  =  Pr(I  I  ui,  u2)  Pr(ui,  u2)  =  Pr(I  I  u,,  u2)  Pr(u2 1  u0  Pr(ui) 

=  0.9  x  0.9  x0.33  =  0.2673 

Pr(I,  di,  s2)  =  Pr(I  I  di,  s2)  Pr(di,  s2)  =  Pr(I  I  di,  s2)  Pr(s2 1  di)  Pr(di) 

=  0.4  x  0.5  x0.33  =  0.066 


The  above  illustrated  probabilistic  evaluation  of  action  alternatives  is  part  of  the  Bayesian 
operators  to  be  applied  in  the  submarine  tactical  decision  making  under  a  number  of  specific  state 
determinations.  The  computation  is  conducted  on  the  basis  of  distinguishing  different  states  for 
taking  different  decision  strategy.  The  computation  can  be  carried  out  by  taking  considerations  of 
the  state  determinations  such  as  (1)  open-loop  where  the  contact  has  no  knowledge  about 
ownship,  so  will  act  in  its  scheduled  path  and  won’t  count-reacting  to  ownship’s  maneuver,  or  (2) 
closed- loop  where  contact  also  known  ownship  and  will  counter-react  to  ownship’s  actions. 

The  idea  of  Bayesian  game  is  to  construct,  for  any  information-incomplete  game  G,  some 
information-complete  game  G*  that  are  game-theoretically  equivalent  to  G  [Harsanyi  1967].  The 
approach  involves  introducing  some  random  events  (variables)  e;,  f;,  i  =  1,  ...  N,  assumed  to  occur 
before  the  players  choose  their  strategies.  The  random  event  e,  will  determine  player  i’s  cost 
function  and  other  resources;  and  so  will  completely  determine  the  payoff  function  U  in  the  game. 
On  the  other  hand,  the  random  event  f;  will  determine  the  amount  of  information  that  player  i  will 
obtain  about  the  cost  functions  and  other  resources  of  the  other  player  j  (j  ^  i),  and  will  thereby 
determine  the  actual  amount  of  information  that  player  i  will  have  about  player  j’s  payoff  function 
Uj.  Both  players  will  be  assumed  to  know  the  joint  probability  distribution  R*({  e,  (.  {  f,  }).  But, 
each  player  will  know  only  his  own  cost  functions  and  resources  but  will  not  know  those  of  his 
opponent;  and  that  he  will,  of  course,  know  how  much  information  ha  himself  has  about  the 
opponent  but  will  not  know  wxactly  how  much  information  the  opponent  will  have  about  him. 

An  important  criteria  for  this  system  is  the  extent  to  which  human-agent  interactions  support  the 
human  operators  performance  and  effectiveness.  Therefore,  it  is  important  that  the  way  that  the 
agents  present  information  (in  their  advisory  capacity)  should  be  adapted  to  the  human  operators 
needs  and  goals.  Therefore,  several  different  types  of  formats  need  to  be  made  available  to  meet 
the  needs  of  the  uses. 
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4  Agent  Architecture  for  the  Hybrid  Model  of  Decision  Support 


4.1  Agent-based  system  architecture 

We  start  with  a  description  of  different  components  of  our  solution  for  tactic  decision  aid  under 
overwhelming  information  inflow  and  uncertainty.  Figure  2  below  shows  an  initial  visual 
representation  of  the  major  system  components  between  the  submarine  unit  command  office  and 
an  Intelligent  Agent  (IA)-based  knowledge  acquisition/decision  support  system. 

Action 


Intelligent 

recommendations 

Unit 

w 

Agent 

"  Objectives  and 

Commander 

priorities 


Figure  2.  Intelligent  Agent  model  of  automated  knowledge  acquisition  and  decision  support 

The  need  for  an  Intelligent  Agent  (IA)  is  manifest  because  the  various  tasks  to  be  performed  by 
the  function  block  (information  gathering,  analysis,  fusion,  evaluation,  reasoning,  etc.).  Game 
theory  needs  to  be  incorporated  into  the  IA  so  that  it  can  be  applied  during  the  reasoning  stage  to 
produce  better  plans  and  action  choices.  There  are  four  major  components  in  the  Intelligent  Agent 
as  shown  in  figure  3.  These  components  are:  (1)  Information  Manager,  (2)  Reasoning  Engine,  (3) 
Adaptation  module,  and  (4)  State  Control  module. 


Figure  3.  Block  diagram  of  the  decision  support  system 


The  State  Control  module  constantly  evaluates  the  available  information  about  contacts  and 
computes  the  probabilities  on  each  of  the  objectives  such  as  whether  own  unit  is  detected  by  the 
contacts  or  not.  A  set  of  states  is  defined  to  span  the  possible  states  of  engagement  and  include, 
for  example,  initial  detection/response,  contact  trail,  track  contact,  close  contact  range,  contact 
avoidance,  contact  prosecution,  etc.  The  Agent  upon  the  detection  of  a  new  contact  starts  the 
algorithm  in  the  initial  detection/response  state  where  the  overriding  concern  is  to  prevent 
collisions  with  contacts  and  conduct  a  maneuver  to  place  Ownship  into  the  optimal  tactical 
advantage.  Based  upon  the  priorities  selected,  the  state  will  change  such  that  the  actions 
recommended  by  the  Agent  are  selected  to  optimize  the  desired  outcome.  The  mathematical 
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problem  is  subject  to  Stochastic  Control  Theory.  The  State  Control  module  determines  the  state 
of  the  encounter  through  the  use  of  Discriminant  Analysis  [Nath  1992].  Discriminant  Analysis  is 
a  standard  statistical  methodology  that  uses  multiple  measurements  of  an  object  or  situation  and 
classifies  it  as  one  of  several  categories.  From  this  data,  the  Agent  will  analyze  the  situation  to 
recommend  the  optimal  course  of  actions  to  achieve  the  goals.  This  optimization  spans  all 
possibilities  and  is  computationally  intensive.  If  however,  realistic  constraints  are  considered  a 
heuristic  model  of  game  theory  will  provide  the  real  time  recommendations  necessary  to  the 
Commanding  Officer.  Our  Agent  will  employ  a  stochastic  game  model  and  the  optimization 
problem  will  be  handled  using  Stochastic  Control  Theory  based  on  Bayesian  probabilities. 

If  the  Contact  changes  course  in  a  manner  such  that  the  probability  of  counter  detection  is 
satisfied,  the  Agent  will  alert  the  Commanding  Officer  and  evaluate  the  situation  to  provide  an 
optimal  recommendation  to  meet  the  goals  as  provided.  We  must  be  able  to  distinguish  a  “true” 
change  in  behavior  from  a  noisy  set  of  data,  and  this  has  to  be  done  in  the  fastest  possible  way. 
This  problem  is  subject  to  Sequential  Analysis  and  is  known  as  the  Change  Point  Problem  [Ghosh 
1991].  An  alternative  approach  would  be  to  use  Discriminant  Analysis  again,  or  a  combination  of 
these  methods  considered  and  evaluated  over  a  period  of  time.  The  Agent  will  continue  to  provide 
recommendations  to  place  Ownship  into  the  best  tactical  position. 

The  Information  Manager  plays  the  important  role  of  interfacing  the  Ownship’ s  information 
sources  to  the  Agent.  Data  must  be  filtered  and  pre-processed  so  that  it  is  in  a  form  that  can  be 
effectively  used  by  the  agent.  The  Preprocessing  and  Filtering  operators  are  in  charge  of  clearing 
up  the  signals  received  from  sensors  and  compensating  for  the  uncertainty  contained  in  the  raw 
information.  This  interface  will  be  standardized  such  that  its  application  can  be  ported  to  all 
classes  of  submarine  missions  with  minimal  installation  and  special  interface  cabling. 

The  Adaptation  module  is  an  important  component  that  collects  the  data  describing  all  the 
contacts  and  analyzes  it  to  improve  the  stochastic  models,  to  improve  the  estimates  for  our 
models,  to  study  the  distributions  of  random  factors  and  white  noise  factors.  Various  statistical 
methodologies  and  data  mining  techniques  will  be  applied  here.  The  adaptive  system  module  will 
apply  previously  captured  knowledge  through  experience  or  decoded  from  known  submarine 
operation  rules  to  initialize  the  parameters  of  the  differential  game  model  and  Bayesian 
probabilities.  It  can  also  make  dynamic  updating  of  these  parameters.  By  activating  the  adaptive 
module,  the  a  priori  probabilities  and  the  conditional  probabilities  of  the  Bayesian  decision  model 
are  established,  obtained,  and  refined.  The  presence  of  the  Adaptive  Component  in  our  system 
architecture  would  allow  us  to  improve  the  performance  of  the  Agent  with  time. 

The  reasoning  engine  applies  Bayesian  and  differential  game-theoretic  operators  to  solve  the 
optimal  control  problem  for  the  submarine  operations  in  various  states.  In  our  mixed  strategy  of 
decision  aids,  the  solution  to  these  states  is  reduced  to  solving  a  set  of  differential  equations 
associated  with  probability  evaluations.  The  process  is  a  result  from  the  optimization  problem 
constrained  to  the  set  of  stochastic  kinematical  differential  equations  describing  the  types  of 
contacts.  A  number  of  related  problems  have  been  solved  by  similar  means,  for  example  maritime 
collision  avoidance,  missile  chasing  a  plane,  simultaneous  confrontation  of  two  planes  etc. 
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4.2.  AEDGE™  Implementation 

The  Submarine  Tactical  Decision  Aid  is  implemented  using  21CSI  Inc.’s  Agent  Enhanced 
Decision  Guide  Environment  (AEDGE™).  The  AEDGE™  product  is  based  on  an  extensible 
architecture  and  a  number  of  standard  components  that  enable  simulation  and  decision  support 
capabilities.  This  marriage  of  the  Decision  Aid  to  a  COTS  based  decision  support  architecture  will 
ensure  standardization  and  portability  of  the  decision  aid  throughout  its  development  [Petrov 
2000,  Hicks  2001].  Below  is  a  brief  outline  of  the  AEDGE™  and  capabilities. 

The  AEDGE™  employs  a  well-modeled  class  and  object  hierarchies  (multiple  inheritance)  to 
support  active  entities  (agents  of  different  types,  simulation  objects  as  well  as  functional  objects) 
communicating  over  a  software  bus,  cooperating  and  so  on.  Services  and  libraries  to  support 
various  analyses  tasks  in  decision  aid  are  provided.  The  system  was  implemented  in  Java™,  with 
Java  Database  Connectivity™  for  DB  access,  Java  AWT  and  possibly  3D  for  interfaces,  JFC  for 
common  objects  and  so  on.  The  kernel  of  the  architecture  consists  of  four  core  and  five  extender 
components.  These  define  the  internal  structures,  dataflow,  and  interface  to  the  architecture.  The 
four  core  AEDGE™  components  are  the  following: 

•  Master  Server.  Tracks  components  and  matches  service  providers  with  service  requesters  and 
facilitates  connections  and  interactions  among  the  rest  of  the  AEDGE™  components.  It 
provides  component  registration  and  tracking  services,  interface  matching  services  and 
component  search,  identification  and  connection  services.  The  Master  Server  is  also 
responsible  for  synchronizing  simulation  time  (and  real  time)  among  multiple  simulation 
servers  and  live  links. 

•  Entity  Representation  Framework.  Provides  the  basic  entities  and  events  for  a  time-event 
simulation  or  live-feed  connections.  The  object-oriented  hierarchy  of  entities  represents  a  wide 
range  of  structures,  vehicles,  platforms,  weapons,  and  sensors.  The  Framework  includes  a 
number  of  interfaces,  which  allow  users  to  add  new  entities  with  new  behaviors  or  with 
combinations  of  existing  behaviors. 

•  Agent  Infrastructure.  Provides  the  basic  inter-agent  communication  and  synchronization 
mechanisms,  as  well  as  the  interfaces  for  agents  to  use  other  data  sources,  such  as  simulation 
servers,  live  data  links,  databases,  etc.  A  base  hierarchy  of  agents  is  also  provided  and  it  can 
be  extended  and  customized  for  particular  user’s  need. 

•  Database  Connectivity.  Provides  generic  and  specific  bridges  to  a  number  of  proprietary  and 
public  databases,  such  as  Oceanographic  Data,  Weapons  Effectiveness,  Vehicle 
Characteristics  and  Performance,  Intelligence,  Rules  of  Engagement,  Weather,  etc.  New 
database  connectivity  modules  can  be  added  by  extending  the  provided  database  bridges  and 
implementing  the  connectivity  interfaces. 

In  addition  to  these  kernel  components,  extender  components  define  the  basic  functionality  of 
information  clients  and  servers  and  define  interfaces  for  adding  new  functionality.  These 
components  are,  in  essence,  templates  for  extending  the  platform  with  new  functionality,  while 
maintaining  tight  integration  and  efficient  implementation.  The  following  standard  AEDGE™ 
extender  packages  are  provided: 

•  Simulation  Servers.  Model  a  particular  aspect  of  the  physical  reality  in  terms  of  the  AEDGE™ 
components.  In  other  words,  a  simulation  server  maintains  a  set  of  entities  (e.g.,  contacts), 
their  properties  (e.g.,  velocity,  tonal  information,  maneuverability,  etc.),  and  those  of  the 
environment  and  models  the  interactions  among  those  (e.g.,  tactical  simulation).  A  simulation 
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server  may  potentially  interact  with  all  four  core  components  of  AEDGE™.  It  registers  with 
the  Master  Server  and  posts  its  exported  services  (e.g.  providing  entity  position  information). 
The  server  manipulates  a  set  of  entities  (object  instances)  from  the  Entity  Framework  that 
represent  the  current  view  of  the  world  according  to  that  simulator.  The  simulation  server  may 
interact  bidirectionally  with  agents  from  that  Agent  Infrastructure,  both  providing  information 
about  the  state  of  the  world  and  receiving  recommendations  and  action  requests  from  Agents 

•  Live  Links.  Based  on  sensor  information  and  reflects  the  state  of  the  physical  world  in  real¬ 
time  and  thus,  the  information  flow  in  unidirectional.  The  live  links  may  provide  entity  or  track 
information,  weather  information,  or  any  other  state  or  capability  changes.  The  links  can 
interface  with  all  core  AEDGE™  components,  much  like  the  simulation  servers  can,  with  the 
limitation  of  unidirectional  communication. 

•  Visualization  Clients.  Responsible  for  interactions  with  the  human  users.  They  present 
visual/auditory  data  from  the  AEDGE™  in  a  clear  and  intuitive  manner,  allowing  for  simple, 
yet  powerful  presentations  of  complex  interdependencies  in  the  simulated/sensor  world. 
Visualization  clients  interact  with  all  components  of  AEDGE™  through  bidirectional 
information  flows.  They  receive  information  on  the  simulated  entities,  their  environment  and 
interactions,  as  well  as  on  agent  evaluations  and  recommendations.  The  users’  interactions 
with  the  Visualization  client  provide  feedback,  which  is  then  sent  back  to  the  AEDGE™  core 
components. 

•  Agent  Clients.  Host  one  or  more  Intelligent  Agents,  which  monitor  the  simulated  world,  react 
to  changes  in  it,  and  interact  among  each  other  and  with  human  users  according  to  their 
specific  agent  behaviors.  The  agent  client  receives  information  from  the  AEDGE™  core  on 
the  state  of  the  world  and  sends  back  agent  requests  and  feedback. 

•  Database  Bridges.  Natural  extension  of  the  AEDGE™  core  Database  Connectivity.  Bridges 
to  characteristics  and  performance  data,  weapons  performance  and  effectiveness  data,  and 
terrain  databases  are  provided.  Interfaces  for  new  database  bridges  are  also  provided. 

5  Conclusion 

This  paper  explores  the  problem  of  providing  tactical  decision  support  to  a  submarine  CO  from 
the  perspective  of  two  fundamental  theories  and  their  complementary  roles  in  decision  science:  the 
differential  game  theory  and  the  classic  Bayesian  probability  theory.  The  paper  focused  on  an 
effort  aimed  at  alleviating  the  problems  of  decision  aid  under  uncertainty  through  cooperative 
software  enterprise  (agents)  that  integrates  a  set  of  game  theoretic  modeling  and  stochastic 
reasoning  techniques  for  the  building  of  man  effective  knowledge  acquisition  tool  to  support 
military  unit  tactical  decision  making  in  time-critical  missions.  A  COTS  based  architectural 
framework  was  chosen  and  will  be  implemented  using  2  lCSI’s  Agent  Enhanced  Decision  Guide 
Environment  (AEDGE™).  To  date,  the  culminating  effort  has  been  the  research  of  the  submarine 
tactical  problem  domain  as  well  as  the  theoretical  basis  behind  the  differential  game  and  Bayesian 
probability  theories.  The  system  could  enable  military  analysts  and  planners  within  a  unit  to  enter 
directly  into  a  game  theoretic  decision  support  tool  the  domain  knowledge  such  as  entities  and 
agent-based  rules  about  entity  interaction.  The  decision  support  tool  that  receives  the  new  domain 
knowledge  will  be  able  to  provide  comprehensive  insight  to  the  action/reaction  dynamic  between 
the  deployed  unit  and  potential  adversaries  with  respect  to  the  employment  of  resources. 
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